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Four influential studies on school effectiveness 
reported by Brookover and Lezotte (1979) , Edmonds and 'Frederiksen 
(1979), Phi Delta Kappa (1980) , 4nd Rutter and others (1979) are 
limited in their usefulness a$ recipes for creating effective schools 
by the following four issues: (1) The fout studies differ in their 
definition of what "effectiveness" means. This variety of definitions 
Should alert practitioners to the inadequacy of generalizing from 
these studies and the need to develop their own concept of 
effectiveness tailored to their particular situations. (2) The 
studies' lack of agreement .on which characteristics most contribute 
to school effectiveness, and discrepancies between their conclusions 
a$id specific findings, should encourage practitioners to develop 
their own list of si tuation^tailofeed effectiveness characteristics. 
(3) The reliability of the studies' results is made questionable by 
the use of standardized, norm-referenced tests as indicators Q*f 
academic achievement and by other aspects o^f the studies 1 research 
techniques and strategies. (4) Practitioners should be aware* 3 that the 
Studies indicate correlations rather than causal relations, and thus 
refrain from using the studies as a recipe for creating effective 
schools. They should also heed the studies ' emphasis on the 
importance of characteristics' interaction over any single 
characteristic. (JBM) > 
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THE^FFECTIVE SCHOOLS MOVEMENT: STUDIES, ISSUES, AND APPROACHES 

. x ■ * V 

introduction 

In the early 1970s, educational researchers started systematically 
examining schools in an attempt to dispell a public perception — 
created in p^^by the educational research community , itself 1 — that • 
schools had li'ttle effect on students 1 achievement and success when 
compared to the ef fefcts that their family background and socioeconomic 
situation had. The initial goals of these examinations were to'detrton- 
strate that some schools do have a beneficial effect on students* 
achievement and success and to identify factors controllable in the 
schools that influence that achievement and success 2 . 

For the most part the studies and reports' that resulted from these 
investigations met these goals. Study after study described, schools 
that -were effective and named characteristics which seemed to be asso- 
ciated with that effectiveness. As this evidence seemed tp mount, 
however, there came a shift in emphasis from theory and research to 
policy and practice. The discovery of schools that were effective 
irrespective of the family, social, or economic conditions of their 
students, brought on a movement to use these effective schools and 
their characteristics as models for school improvement. As* this 



1 ' 4 

For example, some of the conclusions reported by Coleman et al. 
19.66; Jencks et al. 1972; and various National Assessment of 
Educational Progress (NAEP) reports, for example, NAEP, 1977 and NAEP 
1979. ■ ' .« 

2 • 

See for example, Brookovefc et al. 1978; California State 
Department of Education, L977; Edmonds," 1977; Rutter et al. 1979: and 
Weber, 1971* { 



movement gained momentum, effective schools studies became so influ- 
ential that they were used, in some cases, as a basis for far-reaching 

educational policy decisions and large-scale school improvement 

3 

initiatives . In short, they were used as recipes fo^ creating effec- 

*• . ■ 

tive schools. 

Recently, however, a number of questions have. been raised about 
effective schools studies. A growing number of authors' are pointing 
out^that, although the studies 1 results seem to make sense and seem to 
be on the right track, there are serious limitations to them. 

What follows is a brief overview of some issues stemming from 
questions raised about four effective schQols studies which, in the 
opinion of jinany, suggest that these studies should npt be used as 
recipes for creating effective schools. It should be noted that no one 
(including this author) believes .that these studies are wrong or poorly 
done. On the contrary, most look at the results and conclusions of 
these studies as providing insights into school effectiveness and 
success that can -benefit school improvement. ^The' issues outlined in 
this overview are nofme.ant. to diminish these contributions. R&ther K 
they are raised as a caution* to practitioners and policy makers against 

using these studies as recipes for creating effective schools. 

# -J ' . 



For example, Ronald Edmonds (1981) has described several large- 
scale initiatives based on effective schools studies undertaken in New 
York City, Milwaukee, St. Louis, New Jersey, and Connecticut. Both 
Dale Mann (1980) and Daniel Levine (in a personal communication, 
December 10, 1980) have asserted that the number of these effective 
schools improvement initiatives seems to be growing. 



. Pour studies of school effectiveness seem to have exerted a grea& 
deal of influence on educators and the results of these studies are the 
ones most typically underlying school effectiveness improvement initi- 
atives. These studies are the ones reported by Brookover and Lezotte 
(1979), Edmonds and Frederiksen (1979>, Phi Delta Kappa (Duckett et al. 
1980), and Butter et al. (1979).. There are four issues, however, 
associated with these studies which limit the degree to which they can 
be used as recipes f>r creating effect^ schools. These issues have 
to do with: 

t 

• the definitions of "effectiveness"; 

■> • • 

• the match between specific findings and general 
x • conclusions; 

• y&e reliability of the results; and 

%. 

• the practicality of using the findings and 
conclusions. " 

Definitions of Effectiveness 

The first issue that limits the usefulness of these studies and 
/that practitioners often overlook when attempting to translate the 
results into improvement initiatives centers on the question, "Whar 
does, 'effectiveness' mean?" In. these four studies, there is a great 
deal of variety in the Way this question is answered. Each set of 
authors seems to have a different conception of an effective school. 
Intact, kach uses a different term to describe it (see * 
Illustration 1). - 



Illustration 1 



Terra 



Improving Schools 



Effective Schools 



Exceptional Schools 



Schools That Differ 



Brookover & 
Lezotte (1979) 



Edmonds & 
Frederiksen (1979) 



Phi Delta 
Kappa (1980) 



Rutter et al. 
(1979) 



Definition 



Schools which between 1974 and 
1976 showed an increase of 
51 or more of fourth grade 
students who could master at 
least 752 of the objectives 
tested by a math and reading 
test wh.il e simultaneously* 
showing a 51 decrease in the 
ones who could only master 
less than 2yt of the same 
objectives. 



Schools where more than half 
of the sixth grade students 
scored at or above the 75th 
percentile on a verbal 
aptitAide' test. 



Schools that showed a positive 
change in any one or a comb^na- 
i tion of: student achievement; 
student attitudes toward school 
or themselves as learners; 
teacher attitudes toward school 
or students as learners; 
community/parent attitudes 
toward school. 



Schools that were different 
in terms of students* exam 
success, attendance, behavior,' 
and delinquency rates* 

i 




Ednufiids afict Frederiksen, for example, define effectiveness rather 
.narrowly, linking it to scores on verbal aptitude tests. Brookover and 
Lezotte see it as a function of improved math and reading test scores. 
The Phi Delta Kappa authors and Gutter et al. view effectiveness more 
broadly than this. The former include student^ teaicher, and* parent 
attitudes while tl>e latter look at attendance, behavior, and " 
delinquency rates in addition to test scores. Even where there seems 
to be some commonality — that is, in the inclusion of 'improved 
academic achievement in each definition — there is still considerable 

... *> , • 

variety. In each study, the, gauge and focus of that achievement are 

r - 

quite different: from percentage increases in reading^aptitude 

\ • • ' . * 

(Edmonds and Frederiksen), to simultaneous increases and 'decreased, in 

math and reading test scores (Brookover and Lezotte) , to passing grades 

qn national 'comprehensive examinations (Rutter et al.) to a pot pcfurri 

of school-determined, standardized, and ct>rriculum-sp6cif ic test 

results (Phi Delta Kappa)., 

The issue has nothing to do with whether any of these definitions 

are better than any others. Nor is there any issue over whether these 

definitions are good ones, or even appropriate ones. The issue is 

whether these definitions have enough in common to be seen and used as 

if they described one Unified concept, "effectiveness. 11 For the most 

part, they do not. * They seem to vary according to what the authors 

think- are important outcomes of schooling. In\ some' cases, they focus 

strdngly on improved basic skills achievement; in other cases, they 
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include other dimensions 'such as enhanced self-concept and high rates 
of attendance. This variety should alert practitioners to avoid trying 
to generalize a state of school effectiveness based on these studies. 

© 

. Rather, ^practitioners should recfognize that school effectiveness may be 
* . art idj^bsyncr&tic concept. It may be different for different schools 
and for different school districts. It may. be focused narrowly on 
basic skills achievement as most effective schools initiatives are . 
Or,** it may go beyond basic skills achievement - to include outcomes which 
many authors believe to be equally important • such as the ability to 
solve problems, apply information, adapt to new social situations, and 



so on.^ "In short, practitioners heed to develop their own concept of 
effectiveness tailored to their particular situations. 
Match Between Findings artd Conclusions 

Each of n the four studies noted above includes a summation, a set 

•ft ^ 

of general conclusions vhich, in effect, describes characteristics of 
"effective" schools*' The issue of match "revolve.s around these charac- 
teristicjs, how well they match from study to study, and hoW accurately 
they reflect the specific findings of the research contained in the 
studies (see\ Illustration 2). 

Although \the characteristics with which these authors conclude 
their studies seem in some cases to be similar to each other, they do. 



^■4 

See for example, the New, Jersey Education Association' s Urban 
Education* Committee report to the Delegate Assembly (May 16, .1981)*, 
Ronald Edmonds 1 .remarks as reported in Report on Education Research 
(Inner City Schools^ J98£L^ and Dale Mann's remarks at the- Third Tri- 
State Conference on Improving Basic Skills-v (1980V. 

5 

See for example, Banks, 1979; Brimer et al. 1978; Eigerman, 1980; 
Lightfoot, 1978. f* 



Study 

CJ»a meter 1st ics 



Brobkover & 
Lezoltte (1979) 



Illustration 2 



Edmonds* 
(1981)& 



Phi l)elta 
Knppd (1980) 



Rutter et al., 
(1979) 



• Improving schools accept 
and emphasize the 
importance of basic skills 
mastery! as prime goals 
and objectives; 

» Staff fcf improving 
schoolfc believe all 
students can master 
the bisic skills 
objectives and they 
believe the principal 
shards this belief; 

Stafff of improving 
schdols expect their 

Pents will go on 
witrfh their education; 

aff of improving 
hoo^s do ntft trake >. 
cuses: they assume 
sponsibility for 
aching basic skills 
/and are committed to do 
so; 



Staff of improving 
schools spend more time 
on achieving basic \ 
skills objectives; 



(continued on - 
next page) 



• Clarity that pupil 
acquisition of the 
basic school skills 
takes precedence over » 
all other school 
activities; 

• A 'climate^of expectation 
irt which no children 
are permitted to fall 
below minimum but 
efficacious levels of 
achievement; * 

• Strong administrative 
leadership without 
which the disparate 
elements of good schooling 
can be neither brought 
together r.or kept 

' together; ' * ^ 

• Presence of a means by 
which pupil progress can 
be frequently monitored; 

• An atmosphere that is 
orderly without being 
rf 8id. quiet without 
being oppressive, and 

^generally conducive to 
the instructional business 
at hand. 



• Successful schools 
are charaf terized 
by clearly stated 
curricular goals and 
objectives; 

• The leaders 1 atti- 
tudes toward urban 
education and expec- 
tations for school 

, or program' success 
determine the impact 6f 

*the leader on excep- < 
tional schools; 

• The behavior of of the 
designated school or 
program leader is crucial 
in determining school 
success; 

• Successful. urban schools 
frequently employ 
techniques of individual- 
ized instruction; 

■ Structured learning 
environments ar« partic- 
ularly successful in 
urban classrooms; 



(continued on next 
page) 



• Outcomes were better in 
schools where teachers- 
expected the children to . ' 
achieve well; 

• Outcomes were better in 
schools that provided 
pleasant working conditions 

. for the pupils; 

• Outcomes were better in 
schools where immediate, 
direct praise and approval were 
the prevelant means of classroom 
feedback; 

• Outcomes were better in schools 
where teachers presented 
themselves as positiv^ role 
models demonstrating punctuality, 
concern for the physical well-* 
being of the* scrfbol building, 
concern for the emotional well- 
being of t;he pupils, and restraint 

, in the use of physical punishment; 

■ Children's behavior was better in 
schools where teachers were 
readily available to he consulted 
by children about problem* and 
where many children were seen. by 
teachers; 

(continued on next page) 
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Illustration 2 
(continued) 



Brookover & 
Ifezotte (1979) 



Edmonds* 
(1981) 



,Phi OelC 
Kappa (1980) 



Rutter et al. 
(1*79) 



Character 1st Ice 



• Principals at improving 
schools are assertive Instruc- 
tional leaders and discipli- 
narians, and they assume 
responsibility for the 
evaluation of the achievement 
Of basic skills objectives; 

• Staff at improving 
schools accept the concept 
of accountability and are 

t involved in developing (or 
- using) accountability mode'ls; 

• Teachers at improving schools are 
not very satisfied or complacent 
about the status quo; 

• There is more parent-initiated 
contact and involvement at improving 
schools (even though the overall 
amount of parent involvement is 
less) ; 

• The compensatory education 
programs in improving schools 
de-emphasize paraprofessional * 
involvement and teacher involve- 
ment in the selection of Comp-Ed- 
bound students. 



• Reductions in adult /child 
' ratios are associated with 

positive school performance; 

• Successful schools are often " 
supported with special project 
funds from federal, state, and 
local sources; 

• Successful urban schools are 
characterized by high levels of 
parental contact with the school 
and parental involvement with 
school activities; 

• Successful schools frequently 
use staff development or in- 
servicetraining programs to / 
•realizeEheir objectives; 

• The greater, the specificity or s 
focus of the training program . 
in terms of goals or processes* 
the greater the likelihood of 
its success; * 

• Resource and facility manipula- 
tions alone are insufficient to 
affect school outcomes. ^ 



Outcomes were better in 
'VjSchOols where a high 
proportion of children 



lit 



held some kind of 
position of responsibility 
in the school system; 



•A school's atmosphere 
,.#8 influenced, posi- 
tively by *the r degree 
AA which it functions 
*fg£<fl coherent,, whole, 
i$pth agreed ways of doing 
[-things which are consist- 
ent throughout the school 
^nid whichUhave . the general 
\1^|iport o^all staff. 



Edmonds' characteristics are drawn f 



rom a f*ater„ report . They do not come from his and Frederiksen , \^979 st^dy. 



1 :. 



I 



11 



not match. The number of c^aracteristfcs is different in each study. 
Also, the .characteristics that Seem similar are* expressed differently. 
^Lastly, some characteristics seen as "indispensable" by some authors - 
for example, strong 'administrative leadership — are not "included at 
all by others. " 

, This absence of match from study, to study poses anf>bstacle for 

practitioners who attempt to use these conclusions' as a recipe. 

Without more unanimity about which characteristics contribute to p 

school's effectiveness, it is difficult to know which characteristics 

t * 
to use as a focus for improvement. And, the studies offer little 

guidance for selecting the most appropriate. 

A more serious obstacle is the low degree of match between some' 

studies' conclusions and ||ir specific findings. ^A. careful comparison 

shows*, that Brookover and Lezott^, Edmonds and Frederiksen, and the Phi 

Delta Kappa authors seem to have done quite a bit of interpretation 

when translating their findings iijto conclusions. 

» For example, both Brookover and Lezotte and the -Phi Delta Kappa 
authors based their conclusions .on specific findings from a variety of 
data sources ~ interviews, questionnaires, ca.se studies, expert 
opinions, and the like. Taken ^a ltqge the r, the specific findings' of 
these different data sources indicate literally dozens of charac- . 
teristics associated with school effectiveness*. Even Recounting for • 
repetition, the specific findings of these studies show many more' 
characteristics than those listed, in the studies* summaries. .Also, the 
characteristics listed as the findings of each data sou'ifce are much 
re specific and detailed than the characteristics in either set of 



conclusions. Moreover, there is no indication in either study how the 
summary statements were distilled from the larger, more detailed body 
of results. The conclusions are interpretations of. the results, hut 
the authors do not explain how they made them. 6 

• Edmonds 4 and Frederiksen present another kind of problem regarding 
the match between specific results and general conclusions. They 
conclude their study with 21 characteristics which describe an- 
eclectic variety of phenomena from number of guidance counselors to 
number of land acres. These apparently reflect their research findings 
and r - in this study - there are no further conclusions. In an 
earlier work, however, Edmonds lists five "indispensable" character- ' 
istics of effective schools, 7 These five characteristics appear again: 
in two more recent discussions. 8 They are the ones listed in 
Illustration 2 and are probably the most well-known characteristics of 
effectiveness. These characteristics are. not the ones that Edmonds fend 
Frederiksen list in their 1979 study. In the .1979 study, they list 
characteristics that are m<w:e specific; also they list more of them and 
they cover more dimensions of schooling. These five do not seem to 



6 More, recently, Brookover (1981) has gone further and named 

Sl2£3%? ?ff f l8tl " of sch00ls ,, e£fecti V e school learning 

climates' divided into three categories — ideology, Organization aL 
instructional practices. Although this new list Saws on both tie ' 
specific findings and general 'conclusions of the 1979 study, some 
characteristics have been dropped (e.g., the one involvSf eacher 
selection of Comp-Ed students) and others added (e.g. ,: presence of 
cooperative team learning). F 

7 Edmonds, 1978. 
g 

Edmonds, 1980 and Edmonds, 1981 V 



come from the longer 1979 list. And, it is unclear what research was ' 
used to arrive at these five characteristics. 

In sum, the degree to which, the overall conclusions of these four 
studies match each other, match their specific findings, and are dfliv- 
$d from the specific findings deserves attention, particularly if they 
are to be used as a basis for* creating effective schools. Practi- 
tioners should be aware that these studies show that there ar-e many 
characteristics of effectiveness and that not all of them-jappear in all 
the schools studied. In fact, the studies seem to show that 
"effectiveness" involves many diverse characteristics. Quite possibly 
these characteristics are idiosyncratic to specific schools. In fact, 
it may be that there is no one set of universally applicable character- 
istics of effectiveness even though some of the same characteristics 
may* apply in different/ situations. When creating effective schools, 
practitioners would do well to uSe the characteristics outlined in the 
studies as a framework. However, they also should look for character- ' 
istics that may go beyond those described in the studies and which may 
have greater influence in their school or district. So in addition to 
developing their own concept of effectiveness tailored to their own 
situation, practitioners should develop their own list of character- 
istics of effectiveness. These too should be tailored to fit the 
situation. , V 

Reliability of Results 

Objections that have been raised about the reliability of these 
studies 1 results also should encourage practitioners to go beyond them 
when attempting to create effective schools. One objection has to do 



with the measurement of academic achievement. Simply stated, there is* 

a question as to whether the scores of the standardized, norm- 

- referenced tests typically used to measure academic/achievement in most 

of these studies are/ adequate indicators of. achievement y Because these 

tests do not necessarily reflect a particular school's individual 

curriculum focus 0r achievement goals and because they are often 

presented as y sch6ol-wide or district-wide averages, many have expressed 

skepticism over their usefulness as realistic barometers of a schools 
9 * 

effectiveness. 

Other objections question aspects of the .studies 1 research tech- 
niques and strategies. 

A number of authors, for example, have noted that the studies use 
methodological approaches which may color/both specific findings and 
general conclusions. Questions over the studies' use of surveys, 
student test results, interviews, and questionnaires as data sources 
have been raised in this regard. Issues of sample size and selection 
procedures also have contributed to the question of the reliability of 

the studies 1 findings. There is even some question whether the maior 

* - 

goal of these studies, isolating a set jof factors of effectiveness,, 
contributes to the unreliability of their findings. 1 ; 1 



See for example, Bridge, Judd, and Mooclc (1979),; Brimer et al. 
(1978); Madaus et al. (1979); and Cuttace (1982). 

10 o 

See for example, Brimer et al. 1978;. Cuttace, 1982; Madaus et 
al. 1979; Rowen et al. 1982; Scott and Walberg, 1979; Tanner and Celso. 
1982; and Walberg, 1982. 

11 . *t ' / 

See for example, Cuttace, 1982; Rowen et al. 1982: arid Walberg. 
1982. ** 



Although in some, cases technical, these issues are not^^trjvial. 
They illustrate the need for a more-than-casual analysis 6f these 
• school effectiveness studies 'before using them as recipes to create an 
effective) school. The issue/s — and the authors who raise them ~ 
suggest that these studies might have had different conclusions had' 
different* test instruments W methodological approaches been used. 
This implies that there may be alternative conclusions about' what makes 

* - An 

a school effective and that there may be alternative ways to make it 

< ■ 

that way. In fact, there are many other studies of student achieve- v 

/.'"'■ \. 
ment, school effectiveness, and* factors that influence both which do \ 

12 - '■' 

draw different conclusions. Practitioners should be. aware of these. 

alternatives and be ready to weigh each improvement option they ^hggest 

carefully before/ selecting which will be the focus of an improvement 

initiative. / 

Practicality, o/f the foldings and Conclusions 



"The final issue associated with these studies is probably the most 
crucial. The issue is whether the results and conclusions of these 
studies can be put into practice and it revolves around questions of 
causality and interaction. 

Over and over again in the course of their descriptions, the 
author? of these studies emphasize that they are .outlining correlations 
(ones that occur at the same time), not causal relations (ones that 



/ 12. 



nj j " See for Example, Benbow and Flaxman, 1980 and 1981; Glasman and 
^iniaminov, 1981; Gordon, 1979; and Walberg, 1982. 



1 Q 

make each other happen) . : The significance of this distinction is an 
important one for practitioners. It means that these studies' results 
and conclusions should not be interpreted as a recipe for creating an 
effective school for the authors themselves cannot be sujre that a 
school is effective because it has the characteristics described. 14 
Most of the authors are candid about this and all warn against using 
the findings as a recipe. It is a warning practitioners should not 
ignore. 

• Practitioners should not ignore another significant caveat from 
the authors of these studies — a caveat about the interaction of ; : 
characteristics. In outlining their results all of the authors either 
state or suggest strongly that it is the interaction of characteristics 
that leads to Sigher stude\it achievement and success and to school 
effectiveness. Somfe go even further, noting a number of 
characteristics that were not included in their studies that they feel 
are involved in this interaction. 15 . 

This stress on the importance of interaction adds another p^Xem for 
13 

Rutter et al. 1979 say their results show correlations that 
"infer" causal relations (p. 181). This statement has. been the 
source, of much of the criticism of their work (See for example, 
Cuttace, 1982). 

14 : ' 
Some like Rowen et al. 1982 even suggest the opposite: that 

certain characteristics exist because the school is effective or 

because student achievement is high. 

15 V 
Rutter et al. mention a balance between academically able and 

academically less able student populations. BrbokoVer (1981) mentions 
cooperative team learning. Edmonrfs (1981) alludes to interest, 
support, and assistance from state or district educational agencies and 
their personnel. 
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practitioners. In effect iK^ntrdduces a new characteristic of effec- 

» • 

tiveness — the interaction itself. Although all of the studies say ° 
this interaction is critical to school effectiveness, none are specific 
about the nature of this interaction. Moreover, none offer any guid- 
ance for helping practitioners develop it or take advantage of it. 

Yet it is fairly clear from these studies that a synergistic 
approach is the best one for creating an effective school. 7 That is, 
practitioners must look not only for the impact of individual charac- 
teristics.on their school's effectiveness, but also for the way those 
* characteristic work together. And, they must not focus only on maxi- 
mizing the influence of each characteristic but- rather on maximizing 
the influence of all the characteristics as' they interact to result in 
v an elective school. 



' Conclusions 

Effective schools studies have provided educational practitioners 
and policy makers with nearly overwhelming evidence that effective 
schools exist. They have described many schools that probably influ- 
ence student achievement and success as much. as or more than family 
background and socioeconomic situation. But they have not provided 
practitioners with a recipe for creating effective schools. 

School effectiveness seems to be a complex phenomenon and creating 

a 

^effective school is likely to be a complex process, ^t is not one 
that can^be undertaken by following a recipe. The effective schools . 
studies are useful, however, because they provide a framework which 
practitioners can combine with their own knowledge about their own 
schools to: 

! . ' ' ' - • 15 



o determine the best definition of effectiveness for a 
school; 

o decide which school outcomes are or should be 
associated with the definition; 

o selVct school-focused indicators of success that 
reflect the definition for each outcome; 

o assess the school to identify characteristics that 
promote this success or prevent it and determine how 
the characteristics influence each other; and 

o develop a school-based plan to maximize, the charac- 
teristics as they interact to promote school 
effectiveness, ' s - 

t Effective schools seem to exist. Hundreds of them have been 

described. Each one's effectiveness, however , seems to represent an 

intricate -.- perhaps idiosyncratic -- phenomenon which, 4 in turn, is 

probably the result, of intricate' — perhaps idiosyncratic — processes. 

As yet, there *re no recipes for creating effective schools, 

V ' ' • 
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